Digital cameras have largely replaced analog and film cameras in many applications, including (but not limited to) regular consumer (amateur and professional) photography, cell-phones, medical imaging, remote sensing and security monitoring. Digital cameras use image sensors (also referred to as “area array detectors” or “imaging elements”) with millions of pixels per chip, typically of either charge coupled device (CCD) type, complementary metal-oxide-semiconductor (CMOS) type, focal plane array (FPA) type or micro-bolometer type. There are various known types of CCD, CMOS and FPA image sensors. In particular, CMOS sensors may be of active pixel, passive pixel or digital pixel type. Active pixel CMOS sensors can incorporate data conversion and processing in addition to photon detection on the same chip, leading to so called “camera-on-chip” systems, see e.g. “Active Pixel Sensor Design: From Pixels to Systems” by A. Fish and O. Yadid-Pecht” in “CMOS Imagers: From Phototransduction to Image Processing”, ISBN 1-4020-7961-3, Kluwer Academic Publishers (hereinafter “Fish”). Most image sensors have microlenses built on their pixels to increase light capture. For simplicity, an image sensor is referred to hereinafter simply as “sensor”.
A block diagram of a typical digital camera system 100 is shown in FIG. 1. The figure is taken from Fish. The figure shows the difference between the building blocks of a CCD camera and of a CMOS camera-on-chip. A CCD camera typically includes a lens 102, an on-chip or off-chip color filter 104, a sensor (CCD) 106, an analog signal chain (ACS) 108 for amplification and a control block 110 for controlling the functions of the sensor and the ACS. A CMOS camera-on-chip includes the same elements (except that sensor 106 is an active CMOS sensor) plus an analog-to-digital converter (ADC) 112 and a digital signal processor (DSP) 114, both coupled to the control block. In the prior art, one may also find block diagrams with elements that are given somewhat different names, but which in essence refer to the same functionalities.
Spectral imaging (SI) systems (also referred to herein as “spectral imagers” or “SI cameras”) are well known in the art. Their main function is to access images of an object or scene separately in several spectral ranges (wavelength or “spectral” bands) of the spectrum. The number of spectral bands substantially exceeds the three RGB (red-green-blue) bands typical for state-of-the-art color digital cameras. Acquisition of the full spectrum enables recognition of spectral signatures of chemical substances and their position within the object. Furthermore, an image of the object in any selected spectral band can uncover hidden properties of the object that may be important to an application. Depending on the number of spectral bands used, spectral imagers are referred to as being either multispectral (MS) or hyperspectral (HS). The differentiation between “MS” and “HS” is somewhat arbitrary, one definition having “MS” including up to 10 spectral bands and “HS” as having more than 10 spectral bands.
Spectral imaging involves acquisition of a three-dimensional “data cube” (also referred to as spatial-spectral cube, spectral cube, object cube, image cube or hypercube) having two spatial dimensions and one spectral dimension. The SI principle of operation is widely known (see. e.g. Y. Garini et al., “Spectral Imaging: Principles and Applications”, Cytometry Part A, 69A 735-747 (2006)) Most spectral imagers acquire simultaneously only two out of the three data cube dimensions. The third dimension is acquired sequentially (i.e. by scanning). They can be broadly classified (see M. F. Carlsohn, “Spectral image processing in real-time”, J. Real Time Image Processing, vol. 1, p. 25-32, 2006) as either “staring” imagers or “pushbroom” imagers. In staring imagers the simultaneous acquisition is of both spatial dimensions, with the spectral dimension acquired sequentially (wavelength scanning). In pushbroom imagers, the simultaneous acquisition is of the spectral dimension and of one spatial dimension, with the second spatial dimension acquired sequentially (spatial scanning). In both cases, the data cube is acquired sequentially in time (temporal scanning). Neither staring nor pushbroom imagers are particularly relevant to the present invention.
The SI class most relevant to the present invention is that of so-called “snapshot” imagers. These imagers perform simultaneous and instantaneous (“snapshot”) acquisition of all three data cube dimensions, without any type of scanning. Most snapshot SIs are based on non-scanning computed tomographic imaging spectrometer (CTIS) designs, see e.g. M. Descour and E. Dereniak, Applied Optics, Vol. 32, p. 4817-4825, 1995. CTIS designs use one of two dispersing element (“disperser”) types: a rotating disperser (prism or grating) or a two-dimensional grating disperser (e.g. in U.S. Pat. No. 6,522,403 to Wilson and U.S. Pat. No. 7,092,088 to Schau). A block diagram of a typical CTIS system 200 (taken from Schau) is shown in FIG. 2. System 200 includes foreoptics 202, a disperser 204, a field lens 206 and a FPA 208. The output of the FPA is processed in a processor (DSP) 210. A CTIS-based SI camera is being advertised by SnapShot Spectra, 974 East Elizabeth, Pasadena, Calif. 91104, USA, and has been successfully used in ophthalmology (W. R. Johnson, D. W. Wilson and W. Fink, “Snapshot hyperspectral imaging in opthalmology”, J. Biomedical Optics, 12(1), 014036, January/February 2007).
All CTIS designs inherently require a plurality of diffraction orders to be provided by the rotating or two-dimensional grating disperser. Major disadvantages of CTIS-based SI cameras include the need to have a field stop and to form an intermediate image, thereby leading to necessarily more complicated and expensive optics.
Other types of snapshot SI imagers, e.g. as disclosed in US patent application 20060072109 by Bodkin, use a lenslet and/or pinhole array to divide a field of view into multiple channels, which in turn are dispersed into multiple spectral signatures and observed on a two-dimensional FPA in real time.
Key disadvantages of existing snapshot spectral imagers include complicated and expensive optical setups, some requiring moving parts, a lack of miniaturization leading to relatively large sizes and high cost. The smallest and least expensive spectral imagers (of any type) are still significantly larger and more expensive (the latter by at least an order of magnitude) than state-of-the-art digital cameras. The factors of size and cost become critical in both consumer markets and in applications requiring miniaturized and disposable imaging systems, for example in-vivo imagers such as the gastro-intestinal (GI) tract capsule (“also referred to as “diagnostic pill”) described in U.S. Pat. No. 5,604,531. GI capsules of this type can provide only simple spatial optical information. A GI capsule functionality may be significantly enhanced by adding spectral imaging capabilities to perform a so-called “optical biopsy”, i.e. detection of tissue disease such as cancer using spectral imaging. While some spectral imaging GI capsules have been suggested, see US Patent Application No. 20050154277 by Tang et al., their optical set-ups are complicated, requiring many sub-elements, and consequently difficult to implement in the very restricted space available in such capsules.
To summarize—the state of the art in SI cameras is very far from making such cameras a commodity consumer mass product like regular digital cameras. SI camera miniaturization is even further away from realization. There is therefore a need for, and it would be advantageous to have inexpensive and simple to operate snapshot SI cameras. It would be particularly advantageous to be able to adapt or convert a digital camera into a snapshot SI camera without significantly increasing its size, complexity or cost. It would be further particularly advantageous to have miniaturized snapshot SI cameras that can be used in GI capsules, endoscopes, cell-phones, PDAs and similar applications. Some of these advantages may be realized by adding on-chip spectral data acquisition and processing capabilities to sensors, in particular CMOS sensors, opening the way to “SI camera-on-chip” systems.